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Improving Availability and Scalability 
in Clustered Application Servers 

Field of the Invention 

The present invention relates to a method and means of increasing the availability within a 
5 multitude of application servers providing services to a multitude of application clients. More 
particularly, the invention relates to a method and means of increasing the availability and 
scalability by workload balancing and distribution of workloads to improve resource consumption. 

Background of the Invention 

Enterprises depend on the availability of the systems supporting their day-to-day operations. A 
iffO system is called available if it is up and running and is producing correct resuhs. In a narrow 
, 2 sense, availability of a system is the fraction of time it is available, MTBF denotes the mean time 
'[}^ before failure of such a system is denoted, i.e. the average time a system is available before a 
m failure occurs (this is the reliability of the system). Ideally, the availability of a system is L Today, 

a system can claim high availabiUty if its availability is about 99.999% (it is called fault tolerant if 
H5 its availability is about 99.99% ). J. Gray and A. Reuter, "Transaction processing: Concepts and 

Techniques", San Mateo, CA: Morgan Kaufmann 1993 give further details on these aspects. 

Availability of a certain system or application has at least two aspects: in a first, narrow 
significance it relates to the question, whether a certain system is active at all providing its 
services; in a second, wider significance it relates to the question, whether this service is provided 
20 in a timely fashion offering a sufficient responsiveness. 

As outlined in forther details by D. Loshin, "High performance computing demystified", Academic 
Press, Inc., 1994 and K. Hwang, Advanced computer architecture: Parallelism, Scalability, 
Programmability, PMcGraw-Hill, Inc., 1993 and J. Gray and A. Reuter, "Transaction processing: 
Concepts and Techniques", San Mateo, CA: Morgan Kaufmann 1993 one fundamental 
25 mechanism to improve availability is based on "redundancy": The availability of hardware is 
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improved by building clusters of machines and the availability of software is improved by running 
the same softvv^are in multiple address spaces. 

With the advent of distributed systems, techniques have been invented which use two or more 
address spaces on different machines running the same software to improve availability (often 
5 called active replication). Further details on these aspects may be found in S. Mullender, 
"Distributed Systems", ACM Press, 1993. In using two or more address spaces on the same 
machine running the same software which gets its request fi*om a shared input queue the technique 
of warm backups is generalized by the hot pool technique. 

C. R. Gehr et al, "Dynamic Server Switching for Maximum Server Availability and Load 
10 Balancing", U.S. Pat. No. 5,828,847 teaches a dynamic server switching system relating to the 
if: narrow significance of availability as defined above. The dynamic server switching system 
^2 maintains a static and predefined list (a kind of profile) in each client which identifies the primary 
[ J server for that client and the preferred communication method as well as a hierarchy of 
i y successively secondary servers and communication method pairs. In the event that the client does 
15 not have requests served by the designated primary server or the designated communication 

method, the system traverses the list to ascertain the identity of the first available alternate 
:.Sr server-communication method pair. This system enables a client to redirect requests fi-om an 
L 1 unresponsive server to a predefined alternate server. In this manner, the system provides a 
'^'-^ reactive server switching for service availability. 

20 In spite of improvements of availability in the narrow sense defined above, this teaching suffers 
firom several shortcomings. Gehr's teaching provides a reactive response only in case a primary 
server could not be reached at all There are no proactive elements which prevent a client requests 
service fi:*om a non-responsive server. As the list of primary and alternate servers is statically 
predefined, there may be situations in which no server could be found at all or in which a server is 

25 found not before several non-responsive alternate servers have been tested. Moreover, Gehr's 
teaching does not allow for a dynamic workload balancing improving the availability in the wider 
sense, i.e. the responsiveness. According to Gehr, different clients might be controlled by different 
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lists of servers, which allow for a rudimentary and static workload balancing as different clients 
might send their requests to different servers. In a highly dynamic, worldwide operating network 
situation, where clients and servers permanently enter or leave the network and where the access 
pattern to the servers may change from one moment to the next, Gehr's teaching to improve the 
5 responsiveness is not adequate. 

Another area of technology to be mentioned is the area of Transaction Processing monitors (TP 
monitors). TP monitors have been invented more than three decades ago to make effective use of 
expensive system resources (J. Gray and A. Reuter, "Transaction processing: Concepts and 
Techniques", San Mateo, CA: Morgan Kaufmann 1993): Ever increasing numbers of users liad to 
10 be supported by a system, and it turned out that native operating system functionality did not 
suffice to allow this. A TP monitor as a layer on top of the operating system manages system 

2 resources at a much finer granularity, assigns them with care, and only if needed and only for the 
duration needed. As a result, for one and the same machine and operating system, a given 

; y application can support orders of magnitudes of more users when implemented in a TP monitor 

ljl5 than when implemented based on native operating system features. 

h= The very complex and sophisticated TP monitor technology is primarily limited to a certain server 
only and thus does not solve the availability problem of a distributed network of application 
servers. 

With the advent of distributed systems supporting middleware, object request brokers (be it a 
20 CORE A implementation, or DCOM, or based on the Java beans model) favor commodity cluster 
environments, i.e. environments which are composed out of relatively cheap hardware; refer for 
instance to G.F. Pfister, In search of clusters - 2nd edition (Prentice Hall PTR, 1998), In such 
environments, the service providing software components are simply replicated on multiple 
machines to ensure scalability. But this requires a mechanism to assign service requests to the 
25 various service providers ensuring the effective exploitation of the cluster resources. As a 
consequence, the middleware implementing the distributed system has to deal with similar 
problems as traditional TP monitors did before (by the way, this is one of the reasons why such 
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systems are considered as "TP monitor like systems" today). In fact, the middleware is 
representing a single and central TP monitor. 

These "TP monitor like systems" are much too complicated to develop as well as to administer. 
Moreover they themselves consume a significant amount of processing resources. 

Despite of all of this progress, fiirther improvements are urgently required supporting enterprises 
in increasing the availability of their applications and allowing for instance for electronic business 
on a 7 (days) * 24 (hour) basis; due to the ubiquity of worldwide computer networks at any point 
in time somebody might have interest in accessing a certain appUcation server. 

Summary of the Invention 

An object of the invention is to increase the availability of a multitude of application servers 
providing services to a multitude of application clients. 

It is a further object of the invention to increase the availability by providing a technology, which 
is highly responsive to dynamic changes of the availability of individual application servers within 
the network and which supports a less centralized decision process for workload balancing. 

The invention relates to a technology of workload balancing for improved availability within a 
multitude of applications-servers and a multitude of application-clients interconnected with said 
appUcation-servers by a communication network. 

A proposed method comprises a first-step, wherein an application-client is caching availability 
data of a subset of currently active application-servers as potential target application-servers. 

The method comprises a second-step, wherein for execution of an application-request the 
appUcation-client selects an appUcation-server from the subset based on a load-balancing decision 
of the application-client as target application-server. Finally the application-request is sent to the 
target appUcation-server. 
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of the application-client as target application-server. Finally the application-request is sent to the 
target application-server. 

The proposed technology increases the availability and scalability of a multitude of apphcation 
servers providing services to a multitude of application clients. The current invention provides a 
5 proactive technology as it incorporates in the workload balancing decisions only apphcation 

servers which are active, i.e. which actually are providing their services to apphcation chents. This 
prevents that a client generates erroneous request routings requesting service from non-responsive 
servers. Through caching of availability data of active servers, a dynamic technique with ongoing 
processing is suggested being highly responsive to dynamic network situation where cKents and 
10 servers permanently enter or leave the network and where the access pattern to the servers may 
Pi: change from one moment to the next. Thus, the invention can accommodate hot plug-in of server 

machines into apphcation clusters, thus further increasmg the scalability of the environment. 
1^ Comphcated administration efforts to associate apphcation chents with apphcation servers are 
m: complete avoided. By teaching that the apphcation chents are executing the load-balancing 

decisions a less centraUzed decision process for workload balancing is supported. Significant 
B processing burden is removed from the servers, where, according to the state of the art, the 
L workload balancing decisions would be performed and vdiich typically build the primary 
^i: bottleneck for processing resources. Caching only a subset of active apphcation servers in the 
Q apphcation clients aUows for an efficient usage of the hmited resources on the cKent side: 
10 depending on the size of the subset efficient load balancing decisions can be performed by a chent. 
It does not prerequisite a workload management facility within the environment, which eventually 
also eases the implementation of apphcation servers. 

These and other objects will be apparent to one skilled in the art from the following drawings and 
detailed description of the invention. 

25 Brief Description of the Drawings 
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Fig, 1 is a diagram reflecting the concepts of an application server, a hot pool, an application 
cluster and an application client; 

Fig. 2 is an overview of the basic concepts of the current invention; and 

Fig. 3 visualizes the "watchdog monitoring*' approach of the current invention according to which 
the watchdogs are monitoring not only the current activity status of there associated appKcation 
servers but also are monitoring the activity status of other watchdogs. 

Description of the Preferred Embodiment 

The present invention can be realized in hardware, software, or a combination of hardware and 
software. Any kind of computer system - or other apparatus adapted for carrying out the methods 
described herein - is suited. A typical combination of hardware and software could be a general 
purpose computer system with a computer program that, when being loaded and executed, 
controls the computer system such that it carries out the methods described herein. The present 
invention can also be embedded in a computer program product, which comprises all the features 
enabUng the implementation of the methods described herein, and which - when loaded in a 
computer system - is able to carry out these methods. 

Computer program means or computer program in the present context mean any expression, in 
any language, code or notation, of a set of instructions intended to cause a system having an 
information processing capability to perform a particular fimction either directly or after either or 
both of the following a) conversion to another language, code or notation; b) reproduction in a 
different material form. 

If the current specification is referring to an application it may be a computer program of any 
nature not limited to any specific type or implementation. The terms appUcation cHent and 
application server have to be understood from a logical point of view only relating to some type 
of "instance". These terms do not distinguish necessarily different address space or even different 
computer systems. 
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The current invention is assuming a certain communication path between application chent and 
application server; this does not mean that the invention is limited to a certain communication 
paradigm. 

Also if the current specification is referring to a "database" the term is to be understood in a vv^ide 
sense comprising not only actual databases (like relational, hierarchical databases etc.) but also 
simple files and the like. 

Introduction and Problem 

Enterprises depend on the availability of the systems supporting their day-to-day operations. A 
system is called available if it is up and running and is producing correct results. In a narrow 
sense, the availability of a system is the fraction of time it is available. In a second, wider sense 
availability relates to the question, whether an appUcation service is provided in a timely fashion 
offering a sufficient responsiveness. 

In specific the current invention is relating to environments called "appHcation cluster" based on 
the following concepts which are also depicted in Figure 1 : 

An application server (1 10, 1 1 1 or 1 12) is an executable implementing a collection of related 
services - for instance including access to some shared remote database (100). A hot pool (110, 
1 1 1, 112) is a collection of address spaces each of which runs the same application server and 
each of these appK cation servers receive requests fi-om an input queue (125), which is shared 
between the hot pool members. By a server machine (101, 102 or 103) we mean a certain 
physical machine which hosts a hot pool of appUcation servers. An application cluster (120) is a 
collection of servers which fail independently and each server hosts a hot pool of appHcation 
servers of the same kind. 

Apphcations (130) request services fi'om apphcation servers via application chents. An 
application client (13 1) is an executable which runs on the same machine as the apphcation and 
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which commimicates with a server on behalf of the apphcation. If the communication between the 
application chant and a server is based on (asynchronous) reMable message exchange, the 
apphcation server is said to be message based. In what follows we do assume message based 
communication between apphcation chents and apphcation servers; of course the invention is not 
5 limited to the message based communication paradigm as other paradigms may be used instead. 
Consequently, an apphcation chent requests the performance of a certain service by sending a 
corresponding message into the input queue of a hot pool of associated apphcation servers on a 
particular machine. 

Traditional workload management is often server-centric and very sophisticated, thus, hard to 
10 implement: It takes the particular environment into account, i.e. it is often geared towards the 
f-. underlying operating system or even hardware environment. Thus, the corresponding components 

and even the basic concepts are not portable. These server-centric approaches do not solve the 
i^^^ availabihty problem of a distributed network of apphcation servers. These "TP monitor hke 

systems" are much too comphcated to develop as well as to administer. Moreover they 
^f5 themselves consume a significaiit amoxmt of processing resources. 

1^ As the basic approach the current invention is starting from the observation that in apphcation 
cluster environments, a much less sophisticated workload management suffice because the used 

O machine resources (commodity clusters) are usually much cheaper and thus, a request distribution 
which is to a certain degree uneven (or sub optimal) can be tolerated. 

20 Without workload management performed by the system itself, each chent submits requests to a 
predefined server machine, i.e. an administrator must assign a target machine to each chent: This 
is a cumbersome administrative task, typically resulting in a bad resource exploitation. A typical 
example of this category is the teaching of Gehr et al. as discussed above. 

Furthermore, adding new machines to the apphcation cluster environment requires additional 
25 administrative actions. As the plugged-in machines will receive requests only once associated with 
at least one chent an administrator would have to make corresponding administrative decisions. 
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But such an approach is inadequate in a highly dynamic network situation where cUents and 
servers permanently enter or leave the network and where the access pattern to the servers may 
change from one moment to the next. 

A desired solution to these problems should therefore provide a proactive technology which 
5 automatically determines apphcation servers entering (hot plug-in) or leaving the cluster, which is 
responsive to sudden changes of the access pattern of the multitude of apphcation clients and 
which provide a less centralized decisions process for the balancing problem. 

Basic Idea of the Solution 

The fundamental principle of the current invention is that (elementary but also sophisticated) load 
40 balancing can be performed at the chent site to ease the implementation of apphcation server, i.e. 

apphcation functions; in an ongoing process the server site may provide availability data on 
^ apphcation servers and may thus influence load balancing decisions indirectly, but is not required 
f^Jl to do so. To be more precise, it is the apphcation chent that performs load balancing, not the 
' f application server and not the exploited communication subsystem (e.g. messaging middleware, 
/l5 RPC middleware etc.). Nevertheless, the server site may assist the apphcation chent in workload 
; maaagement. Therefore in view of more advanced embodiments of the current invention the 
pr. current teaching could be described as an apphcation server assisted workload balancing of the 
Z-i apphcation cUents. Based on this principle a less centrahzed decisions process for the balancing 

problem has been introduced. 

20 The further features of the current invention are described by referring to overview given in Fig. 
2. 

Associating Server Subsets To Clients 

For the purpose of associating a certain apphcation chent with a certain subset of application 
servers the apphcation chent (201) caches a chosen subset of server machines as potential targets 
25 for its service requests (202). As indicated by the term cache, this subset of apphcation servers 
may be volatile in nature and may undergo permanent changes. Different mechanisms can be used 
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to determine such a subset for each application client which of course may be different for 
different clients: 

1 . Via profiling each apphcation client can be associated with a fixed subset (203) of 
the multitude of all application servers. From this set the apphcation cUent can 
make its choice of targets. This subset can be fixed for all requests^ or it can vary 
from request to request. Of course the profile (203) may undergo changes while an 
application cKent is active; these changes will then be reflected in the cached subset 
(202). 

2. The apphcation chent can retrieve (204) from a "cluster database" (205) the 
necessary topology information, i.e. which servers build the cluster (209) of active 
apphcation servers (206, 207, 208). This topology lookup can be done when the 
apphcation chent is started, or it can be done for each request (before or after the 
request is issued) or any combination of this. Doing it not just at startup time 
allows to consider even servers that have been hot plugged in recently, i.e. in this 
case also apphcation servers which just became active or inactive will be reflected 
(an aspect which will be discussed further below). 

That (sub)set generally stores availability data of currently active apphcation servers. In the 
simplest case only the identity of the currently active apphcation servers may be comprised. But in 
other embodiments of the invention for instance also the processing power of the individual server 
and/or their current workload or other availability related data may be reflected. 

If the server subset is based on a profiling approach as indicated in (1) above, different 
mechanisms can be used to determine the corresponding subset, for example: 

i. The subsets associated with each chent can be evenly distributed across all 
apphcation chents in (the assumed) case that all apphcation chents produce (on the 
average) the same workload. 

ii. The subsets are determined according to the expected workload generated by a 
chent; for instance apphcation chmts with a large expected workload could be 
associated with apphcation servers with great processing power and/or apphcation 



-10- 



DE9-1999-0'052 



clients with a large expected workload could be associated with subsets with no or 
a modest intersection only. Of course maay other distribution strategies are 
possible. 

If the server subset is based on topology lookup, as indicated in (2) above, a component called 
"server monitor" or "server spray" (210) is introduced. The server spray actually accepts lookup 
requests and retums subsets of servers. Moreover the server spray is monitoring (212) the activity 
status of the apphcation servers within the cluster. Thus, the server spray can be viewed as a 
server assisted workload balancing feature for apphcation chents. The server spray may be located 
on a separate application server (21 1) or may be executing on one of the apphcation servers (206 
- 208), which also provide apphcation services to the chents. In performing the server subsetting, 
the server spray can 

i. randomly choose a subset of (available) servers from the cluster database (called 
simple topology lookup). This approach attempts to generate a workload 
distribution on the multitude of application servers, which in the mean is the same 
on all apphcation servers, 

ii. apply its own load balancing algorithm to figure out the appropriate subset 
retumed to the requesting apphcation chent (called load balanced topology 
lookup). In this case, load balancing is assisted by the server site; nevertheless, this 
does not impact the apphcation chents' load balancing mechanism as it provides a 
decision space within the chents' load balancing mechanism is operating. Existing 
algorithms or components might be reused to implement load balanced topology 
lookup on the chent side. 

Load balanced topology lookup might be considering the number of chents already assigned to a 
particular server, thus evenly distributing subsets of servers across apphcation chents; this might 
be appropriate in case all apphcation chents produce (on the average) the same workload. It might 
be more sophisticated by requiring for instance that each apphcation chent must indicate the 
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expected amount of workload to be generated (for instance by simply specifying a 'load class" 
with the lookup request). The returned subset might be impacted by this specification as it may be 
based on a load-balancing decision of the server spray taking into account the indication of the 
expected amount of workload, the application-servers processing power and/or the 
5 application-servers work load. 

Client-Based Workload Balancing 

The following outline on client-based workload balancing is independent from the mechanism 
used to determine the cached server subset, i.e. it can be based on profiling, simple topology 
lookup, or load balanced topology lookup. It is assumed that the elements of a server subset are 
f|0 "weighted" to reflect processing power and/or any other relevant characteristics relating to the 
n availability of the individual appKcation servers; note, that the case of non- weighted servers is 
covered by assigning the same weight to all servers. 

; Before issuing an application request, an application cUent selects (213) a certain one of the 

- 15 application servers comprised by tiie cached subset (202) as target apphcation sever to send (214) 

the request to. The basic approach of the current invention is that this selection is base on a 
"ll load-balancing decision of said application cUent. This basically realizes client-based workload 
G management. 

Various approaches can be used for this load balancing decision. For instance the application 
20 client could select the target application server randomly from the cached subset. If the availability 
data comprised by the subset stores the "weight" of the servers, for instance expressing the 
processing power and/or the apphcation servers work load (for instance expressed by the depth of 
the input queue, last observed response time etc.), much more sophisticated load balancing 
methods can be used on the client side. Whatever load balancing method is exploited it is 
25 preferably utilized for approximating a even distribution of workload within said subset of 
apphcation servers. 
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In doing so, each application client supports the even distribution of requests across all application 
servers available in the application cluster. Load balanced topology lookup might even ampHfy 
this effect. Ensuring an even distribution of requests results in the exploitation of all available 
resources in the overall environment, i.e. in scalability within the available boundaries. 

5 Hot Plug-In Of Application Servers 

Scalabihty is improved via hot plug-in of additional servers (i.e. application servers, which became 
active, or servers, which became inactive by have been removed from service) and delivering the 
information about new servers to all or selective appKcation cUents. The latter information can 
either be "pushed" (i.e. the server spray is taking initiative to send that information) to the clients 
10 or "pulled" (i.e. the appUcation cHents exphcitly are requesting that information) by them, 

'f t "Pushing" this information in a single burst to all cKents might swamp the network severely 

H impacting the network throughput of "real" requests. This impacts the scalability of the overall 

ffl environment and is thus avoided by the current invention. Instead, the push approach sxiggested 

'X. here might be called lazy multi-casting, meaning that the server spray selects iteratively (with a 

- 15 time delay) subsets of appUcation cUents to which information about the newly added or removed 

u server is sent. Again, much sophistication can be applied in determining the proper subsets of 

% application clients and sequencing of the submission of this information. 

Pulling this information requires that appUcation cUents perform from time to time a topology 
lookup (204): Such a lookup can be done whenever a cUent request has to be processed (before 
20 or after sending an appUcation request), periodically, or randomly. This ensures that whenever a 
new server got added to the appUcation cluster after a certain period of time it will receive service 
requests. In pulling topology information to refresh the server subset in its cache, an appUcation 
cUent fundamentally supports scalability via hot plug-in of servers in commodity clusters. 

When a new server machine is plugged in, the server spray could select collections of logged-in 
25 cUents in a random fashion: It withdraws from their cached server subset randomly chosen servers 
and either pushes the corresponding update into their set or waits until they pull for topology 
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information. If the mechanism used does not store persistently for each application cHent its 
cached server subset, the application cKent will ignore the withdrawal if the server is not within 
the cached set of the appHcation client. Again, to avoid network swamping, only a few application 
chents at a time get the modifications to their server subset pushed. 

5 Maintaining The Set Of Available Servers 

Workload management functions consider the amount of resources available on each server for 
processing cUent requests. On each server, this amount can be perceived as a continuous function 
in time of course depending on the status of work formerly submitted to the server. But these 
functions typically assume that the subject server is up and running in a high availability sense: if 
10 the server failed, these function may produce incorrect results based on the historical status and 
the inherent assumption of "always being up and running". I.e. the workload function might return 

fi. a value from the interval ]0,1 [ although the real value of available resources is "0" because the 
server is not available at all, i.e. it does not provide any computing resources. Thus, by adding 

f-f; fault detection workload management will become more precise. 

= 15 Each hot pool on a server of an appUcation cluster is monitored by a "watchdog". Such a 
[j^ watchdog detects failed hot pool members and recreates them immediately in an automatic 
\1 manner; this is depicted in Fig. 3. The current invention in addition suggests that the collection of 
t7l watchdogs in an appKcation cluster monitor themselves ("watchdog watchdogging" or watchdog 

monitoring). This is achieved via a communication medium that allows to run a protocol to 
20 receive information about the state of a hot pool on a given machine; for instance "heartbeat" 
protocols could be used for that purpose. When one of the server of the application cluster fails 
the watchdog on this server will be detected to have failed (with a latency that depends on the 
monitoring protocol used) by other watchdogs. As a result, the remaining watchdogs will be able 
to multi-cast this fact and send this information to the server spray (which will maintain the cluster 
25 database accordingly) so that the workload functions can make use of this information avoiding to 
schedule requests to the failed server. Similarly, when the server is available again the watchdogs 
wiU detect that. Once the workload functions got that information new requests can be submitted 
to the recreated server. The proposed approach significantly improves the responsiveness of the 
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cluster as it is avoided that application requests are sent to non-responsive servers; moreover 
processing resources are detected as soon as they become available and are included in the 
workload balancing process. 

Thus according to the "watchdog monitoring" approach of the current invention, the watchdogs 
5 are monitoring not only the current activity status of there associated appUcation servers but also 
are monitoring the activity status of other watchdogs. 

Advantages of the Invention 

The proposed technology increases the availability and scalability of a multitude of appUcation 
servers providing services to a multitude of appHcation cHents. The current invention is providing 

40 a proactive technology as it incorporates in the workload balancing decisions only application 

'^J servers which are active, i.e. which actually are providing their services to application cKents. This 
prevents that a client generates erroneous request routings requesting service from non-responsive 

m servers. Through caching of availability data of active servers a dynamic technique and ongoing 
process is suggested being highly responsive to dynamic network situation where cUents and 

d5 servers permanently enter or leave the network and where the access pattern to the servers may 
change from one moment to the next. Thus the invention can accommodate hot plug-in of server 

''Ji; machines into appUcation clusters further increasing the scalability of the environment. 

C CompUcated or due its sheer complexity impossible administration efforts to associate appUcation 
cUents with appUcation servers are complete avoided. By teaching that the appUcation cUents are 

20 executing the load-balancing decisions a less centralized decision process for workload balancing 
is supported. Significant processing burden is removed from the servers, where according to the 
state of the art the workload balancing decisions would be performed and which typicaUy build 
the primary bottleneck for processing resources. Caching only a subset of active appUcation 
servers in the appUcation clients allows for an efficient usage of the limited resources (in terms of 

25 storage and processing power) on the cUent side: depending on the size of the subset efficient load 
balancing decisions can be performed by a cUent. Due to the teachings independence from the 
concrete algorithms for workload balancing the reuse of existing general purpose load balancing 
algorithms are possible. The proposed concepts are portable in the sense that its mechanisms do 
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not require hardware or software specific implementations (like highly specialized TPA Monitors 
do). As a result, it ensures a common behavior across environments. It does not prerequisite a 
workload management facility within the environment, which eventually also eases the 
implementation of appUcation servers. 

5 According to a certain embodiment of the invention, a single cluster database suffices as the only 
centraHzed feature, which in addition can be implemented easily with state of the art technology. 

An embodiment wherein the active apphcation servers are distributed approximately evenly over 
the various subsets of the various apphcation clients turns out to be an efficient approach: such 
subsets can be determined performance-efficiently without increasing workload considerably, 
fip Already an approximately even distribution of the appUcation servers ensures that the probability 
fi. that a servers is overrun is significantly reduced, that the overall throughput to the cluster is 

improved, that availability and responsiveness of the apphcation servers is improved and that at 
m the same time also improved scalability is offered. 

The fiirther embodiment of the inv^tion introducing a server monitor or "server spray'* simphfies 
il5 the implementation of the appUcation cUent. A mere retrieve request suffices to determine the 

subset, complicate database on the client side accesses are avoided, FinaUy the more resource 
C3: expensive monitoring and administering of the complete set of appUcation servers can be executed 

outside an appUcation cUent on a more powerful system; thus the appUcation cUents have to cope 

with the much smaller subset only. 

20 As a farther advantage the server monitor could offer additional assistance if the subset of 
appUcation servers provided is ahready the result of a load-balancing decision. This approach 
allows for a combination of global workload balancing, performed through the server monitor by 
prescribing to the application cUents a "workload control space" (the subset), and a decentralized 
workload balancing, performed by the individual appUcation cUents balancing issued requests 

25 within its assigned subset. 
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An embodiment of the invention wherein an application client sends an indication of the expected 
amount of workload to be generated to the server monitor permits the server monitor to make a 
more precise workload balanced subset selection leading to improved workload balancing 
decisions. Or on the other hand on the client side less sophisticated workload balancing 
methodologies can be applied without effecting the overall availability and scalability. 

As further advantage the appKcation, cUent may perform any type of workload balancing on the 
subset of appUcation servers which may be adapted to the client's processing capabilities. This 
may range from a very economic but nevertheless efficient random selection to a more 
sophisticated workload balancing decision exploiting the application servers processing power ^ its 
current workload or its observed response time. 

A further significant advantage is that the current teaching supports various degrees of the 
actuaUty of the availability data of the cached subset. It may range from a random or periodically 
actualization up to an actuahzation per request. Eventually the lazy-multi-cast approach combines 
high actuality of the availabihty data of the cached subset with a modest load on the network. 

The introduction of watchdogs monitoring the activity status of its corresponding local 
apphcation server which then inform the server monitor, guarantees a decentralized and thus 
efficient monitoring approach. By allowing in addition a watchdog to check the activity of other 
watchdogs, the problem of "checking the checker" is solved. 

While the preferred embodiment of the invention has been illustrated and described herein, it is to 
be imderstood that the invention is not limited to the precise construction herein disclosed, and 
the right is reserved to all changes and modifications coming within the scope of the invention as 
defined in the appended claims. 
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What is claimed is: 



1 1 . A computerized method of workload balancing for improved availability within a multitude 

2 of applications-servers and a multitude of appKcation-cUents interconnected with said 

3 application-servers by a communication network, said method comprising 

4 caching within an apphcation-client, availability data of a subset of currently active 

5 application-servers as potential target apphcation-servers, and 

selecting by execution of an application-request by said application-client, an 
% application-server from said subset based on a load-balancing decision of said 

apphcation-client as target apphcation-server and is sending said appHcation-request to said 
ffi target apphcation-server. 

1 2. A method of workload balancing according to claim 1 further comprising, 



retrieving by said apphcation-client, said subset of currently active apphcation-servers from 
31 a cluster database, said cluster database storing availabihty data of currently active 

T apphcation-servers. 

1 3. A method of workload balancing according to claim 2 further comprising, 

2 choosing said subset of currently active application-servers by approximating an even 

3 distribution of said apphcation-servers with respect to other subsets of other 

4 application-clients. 

1 4. A method of workload balancing according to claim 2 further comprising. 
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2 retrieving said subset of currently active application-servers by issuing a retrieve request to 

3 a server-monitor, said server-monitor monitoring said application-servers activity status. 

1 5. A method of v^orkload balancing according to claim 4 further comprising, 

2 choosing by said server-monitor, said subset of currently active application-servers from 

3 said cluster database by a random selection. 

1 6. A method of v^orkload balaacing according to claim 4 further comprising, 

2 choosing by said server-monitor, said subset of currently active application-servers from 
1^^. said cluster database based on a load-balancing decision of said server-monitor. 

P 7. A method of workload balancing according to claim 6 further comprising, 

2:' choosing by said server-monitor, said subset of currently active application-servers by 

3 approximating an even distribution of said application-servers with respect to other subsets 
of other apphcation-clients. 

0 8. A method of workload balancing according to claim 4 further comprising, 

2 sending by said appUcation-client together with said retrieve request, an indication of the 

3 expected amount of workload to be generated by said application-client, and 

4 basing said server-monitor's load-balancing decision on said indication of the expected 

5 amount of workload and/or said application-servers processing power and/or said 

6 application-servers work load. 

1 9. A method of workload balancing according to claim 1 , 
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2 wherein selecting by said application-client comprises selecting said target 

3 application-server randomly. 

1 10. A method of workload balancing according to claim 1 , 

2 wherein said availability data of said subset of currently active application-servers 

3 comprises said application-servers processing power and in addition said 

4 application-servers work load and/or said application-servers observed response time and 

5 wherein selecting by said application-client comprises selecting said target 

6 application-server approximating an even distribution of workload within said subset of 

7 currently active application-servers. 

fi 11. A method of workload balancing according to claim 1 further comprising, 

dynamically updating said availabiUty data of said subset of currently active 

§^ application-servers by explicitly requesting from said server-monitor an update of said 

4 subset of currently active application-servers, and/or 

BJ; dynamically updating by said server-monitor, said availability data of said subset of 

6J currently active application-servers by receiving an update of said subset of currently active 

T application-servers. 

1 12. A method of workload balancing according to claim 1 1 , 

2 wherein said explicitly requesting from said server-monitor an update of said subset of 

3 currently active application-servers occurs at one of: 

4 prior to an application-request, or 

5 after sending an appUcation-request, or 

6 periodically, or 

7 randomly. 
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13 . A method of workload balancing according to claim 1 1 , 

wherein, in the event an additional application-server has become active or if an active 
application-server has become inactive, said server-monitor takes the initiative of sending 
said update of said subset of currently active appUcation-servers. 

14. A method of workload balancing according to claim 13, 

wherein said server-monitor sending said update of said subset of currently active 
application-servers comprises a lazy-multi-cast including; 

iteratively selecting with a time delay, a collection of application-cUents for said multitude 
of appUcation-clients, and 

sending each appUcation-client of said collection an update of said subse of currently active 
application-servers. 

15. A method of workload balancing according to claim 4, 

wherein said application-servers each comprise a hot pool of one or a multitude of 
application-instances, said application-servers being executed in the same or a multitude of 
address-spaces, and 

wherein for each of said hot pools a watchdog is monitoring said hot pool's activity status, 
and 

wherein each of said watchdogs is informing said server-monitor on the workload of said 
monitored application-servers and its current activity status. 
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16. A method of workload balancing according to claim 1 5 , 



2 wherein at least one of said watchdogs is monitoring the activity status of other watchdogs 

3 and, in the event that said one watchdog detects the imavailability of any one of said other 

4 watchdogs, said one watchdog informs said server-monitor of said corresponding activity 

5 status and said server-monitor updates said cluster database accordingly. 

1 17. An apparatus for workload balancing giving improved availability within a multitude of 

2 applications-servers and a multitude of appUcation-clients interconnected with said 

3 application-servers by a communication network, said apparatus comprising 

means for caching within an apphcation-client, availability data of a subset of currently 

P active application-servers as potential target apphcation-servers, and 

65 means for selecting by execution of an appUcation-request by said application-client, an 

f f application-server from said subset based on a load-balancing decision of said 

§ application-client as target application-server and is sending said apphcation-request to said 

£ target apphcation-server. 

is 18. An apparatus for workload balancing according to claim 17 further comprising, 

2 means for retrieving by said appUcation-client, said subset of currently active 

3 application-servers from a cluster database, said cluster database storing availability data of 

4 currently active apphcation-servers. 

1 19. An apparatus for workload balancing according to claim 18 further comprising, 

2 means for choosing said subset of currently active application-servers by approximating an 

3 even distribution of said application-servers with respect to other subsets of other 

4 appUcation-clients. 
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1 20. An apparatus for workload balancing according to claim 18 further comprising, 

2 means for retrieving said subset of currently active application-servers by issuing a retrieve 

3 request to a server-monitor, said server-monitor monitoring said application-servers 

4 activity status. 

1 21. An apparatus for workload balancing according to claim 20 further comprising, 

2 means for choosing by said server-monitor, said subset of currently active 

3 application-servers from said cluster database by a random selection. 

22. An apparatus for workload balancing according to claim 20, 

% means for choosing by said server-monitor, said subset of currently active 

1^' application-servers from said cluster database based on a load-balancing decision of said 

4 server-monitor. 

tj^ 23. An apparatus for workload balancing according to claim 22 further comprising, 

? means for choosing by said server-monitor, said subset of currently active 

3 application-servers by approximating an even distribution of said application-servers with 

4 respect to other subsets of other application-clients. 

1 24. An apparatus for workload balancing according to claim 20 further comprising, 

2 means for sending by said appUcation-client together with said retrieve request, an 

3 indication of the expected amount of workload to be generated by said application-chent, 

4 and 
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5 means for basing said server-monitor's load-balancing decision on said indication of the 

6 expected amount of workload and/or said application-servers processing power and/or said 

7 application-servers work load. 

1 25. An apparatus for workload balancing according to claim 17, 

2 wherein said means for selecting by said apphcation-client comprises means for selecting 

3 said target application-server randomly. 

1 26. An apparatus for workload balancing according to claim 17, 

X wherein said availability data of said subset of currently active application-servers 

P comprises said application-servers processing power and in addition said 

^= application-servers work load and/or said application-servers observed response time and 

iSj wherein means for selecting by said application-client comprises means for selecting said 

# target application-server approximating an even distribution of workload within said subset 

3 of currently active application-servers. 

27. An apparatus for workload balancing according to claim 17 farther comprising, 

T means for dynamically updating said availability data of said subset of currently active 

3 appUcation-servers by expUcitly requesting from said server-monitor an update of said 

4 subset of currently active application-servers, and/or 

5 means for dynamically updating by said server-monitor, said availability data of said subset 

6 of currently active application-servers by receiving an update of said subset of currently 

7 active appHcation-servers. 

1 28. An apparatus for workload balancing according to claim 27, 
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2 wherein said explicitly requesting from said server-monitor an update of said subset of 

3 cxirrently active application-servers occurs at one of: 

4 prior to an application-request, or 

5 after sending an application-request, or 

6 periodically, or 

7 randomly. 

1 29. An apparatus for vv^orkload balancing according to claim 27, 

2 wherein, in the event an additional application-server has become active or if an active 

3 application-server has become inactive, said server-monitor takes the initiative of sending 
^. said update of said subset of currently active appUcation-servers. 

1^: 30. An apparatus for workload balancing according to claim 29, 

^ wherein said server-monitor sending said update of said subset of currently active 

g application-servers comprises a lazy-multi-cast including; 

If means for iteratively selecting with a time delay, a collection of appUcation-cUents for said 

gl; multitude of appUcation-clients, and 

6 means for sending each appKcation-client of said collection an update of said subse of 

7 currently active appKcation-servers. 

1 31. An apparatus for workload balancing according to claim 20, 

2 wherein said application-servers each comprise a hot pool of one or a multitude of 

3 application-instances, said application-servers being executed in the same or a multitude of 

4 address-spaces, and 
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5 wherein for each of said hot pools a watchdog is monitoring said hot pool's activity status, 

6 and 

7 wherein each of said watchdogs is informing said server-monitor on the workload of said 

8 monitored application-servers and its current activity status. 

1 32. An apparatus for workload balancing according to claim 3 1 , 

2 wherein at least one of said watchdogs is monitoring the activity status of other watchdogs 

3 and, in the event that said one watchdog detects the unavailability of any one of said other 

4 watchdogs, said one watchdog informs said server-monitor of said corresponding activity 
status and said server-monitor updates said cluster database accordingly. 

I- 33 . A computer program product comprising a computer usable medium having computer 

5 readble program code means therein for workload balancing giving improved availability 
It within a multitude of applications-servers and a multitude of appUcation-cUents 

4 interconnected with said application-servers by a communication network, said computer 

5 program product comprising 

p computer readable program code means for caching vvdthin an appHcation-client, 

7 availability data of a subset of currently active application-servers as potential target 

8 apphcation-servers, and 

9 computer readable program code means for selecting by execution of an 

10 appUcation-request by said appUcation-client, an apphcation-server from said subset based 

11 on a load-balancing decision of said appKcation-client as target appKcation-server and is 

12 sending said appUcation-request to said target apphcation-server. 

A. 

1 34. The computer program product for workload balancing according to claim 33 further 

2 comprising, 
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3 computer readable program code means for retrieving by said application-client, said 

4 subset of currently active application-servers from a cluster database, said cluster database 

5 storing availability data of currently active application-servers. 

1 35. The computer program product for workload balancing according to claim 34 further 

2 comprising, 

3 computer readable program code means for choosing said subset of currently active 

4 application-servers by approximating an even distribution of said application-servers with 

5 respect to other subsets of other apphcation-cKents. 



The computer program product for workload balancing according to claim 34 further 
comprising, 



3i computer readable program code means for retrieving said subset of currently active 

4 application-servers by issuing a retrieve request to a server-monitor, said server-monitor 

jS. monitoring said application-servers activity status. 

II 37. The computer program product for workload balancing according to claim 36 further 
"2 comprising, 

3 computer readable program code means for choosing by said server-monitor, said subset 

4 of currently active application-servers from said cluster database by a random selection. 

1 38. The computer program product for workload balancing according to claim 36, 

2 computer readable program code means for choosing by said server-monitor, said subset 

3 of currently active application-servers from said cluster database based on a load-balancing 

4 decision of said server-monitor. 
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1 39. The computer program product for workload balancing according to claim 38 further 

2 comprising, 

3 computer readable program code means for choosing by said server-monitor, said subset 

4 of currently active application-servers by approximating an even distribution of said 

5 application-servers with respect to other subsets of other application-clients. 

1 40. The computer program product for workload balancing according to claim 36 further 

2 comprising, 

^ computer readable program code means for sending by said appKcation-client together 

# with said retrieve request, an indication of the expected amount of workload to be 

P generated by said appUcation-chent, and 

^ computer readable program code means for basing said server-monitor's load-balancing 

f7 decision on said indication of the expected amount of workload and/or said 

application-servers processing pow^er and/or said application-servers work load. 

ft 41. The computer program product for workload balancing according to claim 33, 

2 wherein said computer readable program code means for selecting by said 

3 apphcation-client comprises computer readable program code means for selecting said 

4 target application-server randomly. 

1 42. The computer program product for workload balancing according to claim 33, 

2 wherein said availabihty data of said subset of currently active application-servers 

3 comprises said application-servers processing power and in addition said 

4 application-servers work load and/or said application-servers observed response time and 
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5 wherein computer readable program code means for selecting by said application-client 

6 comprises computer readable program code means for selecting said target 

7 application-server approximating an even distribution of workload vdthin said subset of 

8 currently active application-servers. 

1 43 . The computer program product for workload balancing according to claim 33 further 

2 comprising, 

3 computer readable program code means for dynamically updating said availability data of 

4 said subset of currently active application-servers by expUcitly requesting from said 

5 server-monitor an update of said subset of currently active application-servers, and/or 

^ computer readable program code means for dynamically updating by said server-monitor, 

1^ said availability data of said subset of currently active application-servers by receiving an 

^ update of said subset of currently active application-servers. 

■1 44. The computer program product for workload balancing according to claim 43, 

2- wherein said explicitly requesting from said server-monitor an update of said subset of 

31 currently active application-servers occurs at one of: 
T prior to an application-request, or 

5 after sending an application-request, or 

6 periodically, or 

7 randomly. 

1 45 . The computer program product for workload balancing according to claim 43 , 

2 wherein, in the event an additional application-server has become active or if an active 

3 application-server has become inactive, said server-monitor takes the initiative of sending 

4 said update of said subset of currently active appUcation-servers. 
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1 46. The computer program product for workload balancing according to claim 45 , 

2 wherein said server-monitor sending said update of said subset of currently active 

3 application-servers comprises a lazy-multi-cast including; 

4 computer readable program code means for iteratively selecting with a time delay, a 

5 collection of application-cUents for said multitude of application-clients, and 

6 computer readable program code means for sending each appUcation-client of said 

7 collection an update of said subse of currently active application-servers, 

M 47. The computer program product for workload balancing according to claim 36, 

^ wherein said application-servers each comprise a hot pool of one or a multitude of 

application-instances, said appKcation-servers being executed in the same or a multitude of 

4 address-spaces, and 

% wherein for each of said hot pools a watchdog is monitoring said hot pool's activity status, 

B and 

7 wherein each of said watchdogs is informing said server-monitor on the workload of said 

8 monitored application-servers and its current activity status. 

1 48. The computer program product for workload balancing according to claim 47, 

2 wherein at least one of said watchdogs is monitoring the activity status of other watchdogs 

3 and, in the event that said one watchdog detects the unavailability of any one of said other 

4 watchdogs, said one watchdog informs said server-monitor of said corresponding activity 

5 status and said server-monitor updates said cluster database accordingly. 



-30- 



DE9-1 999-0052 



Improving Availability and Scalability 
in Clustered Application Servers 



Abstract of the Disclosure: 

The invention relates to a technology of workload balancing for improved availability within a 
5 multitude of applications-servers and a multitude of application-clients interconnected with said 
application-servers by a communication network. A proposed method comprises a first-step, 
wherein an appHcation-client is caching availability data of a subset of currently active 
application-servers as potential target apphcation-servers. The method comprises a second-step, 
wherein for execution of an application-request the appUcation-client selects an application-server 
0 from the subset based on a load-balancing decision of the appUcation-client as target 
appUcation-server. Finally the apphcation-request is sent to the target apphcation-server. 
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;i known to me to be material to patentability as defined in Title 37, Code of Federal Regulations, 
Section 1.56. 

I hereby claim foreign priority benefits under Title 35, United States Code, Section 119(a)-(d) or 
Section 365(b) of any foreign application(s) for patent or inventor's certificate, or Section 365(a) of 
any PCT International application which designated at least one country other than the United 
States, listed below and have also identified below, by checking the box, any foreign application for 
patent or inventor's certificate or PCT International application having a filing date before that of the 
application on which priority is claimed. 

Prior Foreign Application(s) Priority Not Claimed 



99122914.7 Europe 18/11/99 

(Number) (Country) {Day/Month/Year Filed) 



English Language Declaration 



(if applicable) 



□ 



(Number) 



(Country) 



(Day/Month/Year Filed) 



□ 



(Number) 



(Country) 



(Day/Month/Year Filed) 



Form PTO-SB-01 (9-95) (Modified) 



P02/REV02 



Patent and Trademark Office-U.$. DEPARTMENT OF COMMERCE 



Page 2 of 3 



1 hereby claim the benefit under 35 U.S.C. Section 119(e) of any United States provisional 
application(s) listed below: 



(Application Serial No.) 


(Filing Date) 


(Application Serial No.) 


(Filing Date) 



{Application Serial No.) (Filing Date) 



I hereby claim the benefit under 35 U. S. C. Section 120 of any United States application(s), or 
Section 365(c) of any PCT International application designating the United States, listed below and, 
insofar as the subject matter of each of the claims of this application Is not disclosed in the prior 
United States or PCT International application in the manner provided by the first paragraph of 35 
U.S.C. Section 112, I acknowledge the duty to disclose to the United States Patent and Trademark 
O Office all information known to me to be material to patentability as defined in Title 37, C. F. R., 
Section 1 .56 which became available between the filing date of the prior application and the national 
or PCT International filing date of this application: 



(Application Serial No.) 


(Filing Date) 


(Status) 

(patented, pending, abandoned) 


(Application Serial No.) 


(Filing Date) 


(Status) 

(patented, pending, abandoned) 


(Application Serial No.) 


(Filing Date) 


(Status) 

(patented, pending, abandoned) 



I hereby declare that all statements made herein of my own knowledge are true and that all 
statements made on information and belief are believed to be true; and further that these statements 
were made with the knowledge that willful false statements and the like so made are punishable by 
fine or imprisonment, or both, under Section 1001 of Title 18 of the United States Code and that 
such willful false statements may jeopardize the validity of the application or any patent issued 
thereon. 
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POWER OF ATTORNEY: As a named inventor, I hereby appoint the following attorney{s) and/or 

agent(s) to prosecute this application and transact all business in the Patent and Trademark Office 

connected therewith, (list name and registration number) 

Floyd A. Gonzalez, Reg. 26,732 

Lynn L. Augspurger, Reg. 24,227 

Lawrence D. Cutter, Reg. 28,501 

William A. Kinnaman, Jr., Reg. 27,650 

William B. Porter, Reg. 33,135 

Andrew J. Wojnicki, Jr., Reg. 

Christopher A. Hughes, Reg. 26,914 

Edward A. Pennington, Reg. 32,588 

John E. Hoel, Reg. 26,279 

Joseph C. Redmond, Jr., Reg. 18,753 





Send Correspondence to: ^^'^y^ ^ ^^"^^'^^ 

IBM Corporation 

2455 South Road, P386 

Poughkeepsie, NY 12601 




-J-: Direct Telephone Calls to : (name and telephone number) 
1=^ Flovd A. Gonzalez, (845) 433-1163 










Full name of sole or first inventor 
Frank Leymann 




Sole or first inventor's signature 




Residence 

Hasenaeckerweg 19, D-71134 Aidlingen, Federal Republic of Germany 




Citizenship 
German 




Post Office Address 
same as above 












Full name of second inventor, if any 
Dieter Roller 




Second inventor's signature 


DatB 




Residence 

Hermann-Loens-Weg 5, D-71101 Schoenaich, Federal Republic of Germany 




Citizenslnip 
German 




Post Office Address 
same as above 
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Application deficiencies were found during scanning: 
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^ □ Page(s) of were not present 

5 for SCanninS. (Document title) 
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□ Page(s) of were not presen: 

for scanning. (Document title) 



□ Scanned copy is best available. 



